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Introduction 

Within primary and secondary school technology education, engineering has been proposed as 
an avenue to bring about technological literacy (Dearing & Daugherty, 2004; Lewis, 

2005). Different initiatives such as curriculum development projects (i.e., Project ProBase and 
Project Lead The Way) and National Science Foundation funded projects such as the National 
Center for Engineering and Technology Education (NCETE) have been developed to infuse 
engineering into primary and secondary education. For example, one key goal of the Technology 
Teacher Education component of NCETE is to impact the focus and content of the technology 
education field at the secondary level (National Center for Engineering and Technology 
Education, 2005). More specifically, the goal is to facilitate students’ learning relative to core 
engineering principles, concepts, and ideas. A number of activities have been developed by the 
center to facilitate these goals, including a series of teacher professional development 
experiences, research designed to identify core engineering concepts, development of 
engineering-related activities, engagement with faculty from the STEM disciplines, and 
interaction with technology education pre-service teachers. 

Through the efforts of NCETE, three core engineering concepts within the realm of engineering 
design have emerged as crucial areas of need within secondary level technology education. 

These concepts are constraints, optimization, and predictive analysis (COPA). COPA appears to 
be at the core of the conceptual knowledge needed for students to understand and be able to do 
engineering design. 

One of NCETE’s five technology teacher professional development institutions (Illinois State 
University) has focused exclusively on the delivery of these COPA concepts. These concepts 
have emerged as distinctly important. At Illinois State University, two cohorts of practicing and 
pre-service technology teachers have engaged in professional development workshops to become 
better prepared to deliver engineering concepts to their students. Through discussions about the 
nature of engineering and engineering design, how it differs from technology education 
processes, and the necessary conceptual and procedural knowledge, constraints, optimization, 
and predictive analysis were identified as necessary for the delivery of engineering design. 

Empirical knowledge is needed to better understand how to increase student learning of COPA. 
This study sought to provide that knowledge using a NCETE cohort of practicing and pre-service 
technology teachers who designed and developed a unit of instruction to deliver these three core 



engineering concepts to secondary level technology education students. Using a mixed method 
quasi-experimental, pre-test post-test no control group design, this study explored the extent to 
which students understood and were able to demonstrate an understanding of constraints, 
optimization, and constraints. It is believed that through this strong conceptual base, a better 
understanding of engineering and engineering design can be achieved. 

Review of the Literature 

Predictive Analysis 

In a review of science, engineering and technology careers. Deal (1994) stated that engineers 
apply mathematical and scientific principles to solve problems. The introduction of these tools 
into the analytical stage of the design process represents an indispensable part of engineering 
design (Harris & Jacobs, 1995). Eekels (1995) observed how the prediction component functions 
in the engineering design process noting that “if the conditional prediction sounds unfavorable, 
then we generally simply abstain from that action and design another action,” (p. 176) which is 
to make the informed decision before constructing the prototype of a design. Hayes (1989) 
observed that predictive analysis is carried out in the planning environment, not the task 
environment, with several distinct advantages: (a) moves made in planning the environment can 
be easily undone while task environment actions cannot be reversed; (b) predictive analysis is 
relatively inexpensive; and (c) it permits design flexibility. 

“The process of thinking before acting” is critical if designing is to be a predictive rather than a 
trial-and-error process (Hayes, 1989). Trial-and-error remains the prevailing approach to design 
in technology education classrooms, where analytical mathematical tools are frequently not used 
to design and prototype design ideas (Lewis, 1999: Merrill, 2001). Lewis (2005) argued that 
conceptual design is within normal purview of technology education and that science and 
mathematics should be taught to help students make predictions about the design through the 
process of analytical design. The Standards for Technological Literacy: Content for the Study of 
Technology (ITEA, 2000) reinforce the systematic aspects of predictive analysis. “Because so 
many different designs and approaches exist to solving a problem, a designer is required to be 
systematic or else face the prospect of wandering endlessly in search of a solution” (p. 91). 

Constraints 

The design processes utilized in engineering and technology education are very similar with 
some notable exceptions. Lewis (2005) has suggested that engineering design places more 
emphasis on assessing constraints, trade-offs, and utilizing predictive analysis compared to 
technology education. The importance of constraints is, however, included in the Standards for 
Technological Literacy: Content for the Study of Technology (2000). In Standard Eight, 
constraints are viewed as an integral part of an iterative process that typically requires students to 
consider costs, economics, feasibility, time, material, and environmental implications. Students 
should be able to assess and incorporate constraints into design activities. 

Addressing constraints early in the problem identification stage may assist students in developing 
viable solutions, since this process helps reduce the size of the solution space (Jin & Chusilp, 
2006). Expert designers typically move quickly from defining the problem (problem space) to 
the solution space by assessing the problem constraints and searching for contextually related 
problems that they have solved in the past (Cross, 2002; Cross 2004; Middleton, 2005), . This is 



similar to a model revised by Middleton, where the problem space is defined as the problem 
state, goal state, and search state. Middleton’s “search state” can be viewed as identifying the 
constraints; while iteratively moving between the problem state and goal state; and concurrently 
decomposing an ill-defined problem into well-defined sub-problems (Cross, 2002; Ho, 2001) 

Optimization 

Design optimization extends beyond simply producing a design that adheres to a defined set of 
constraints or criteria. The purpose of optimization is to achieve the “best” design relative to a 
set of prioritized criteria or constraints. These include maximizing factors such as productivity, 
strength, reliability, longevity, efficiency, and utilization. Engineers must make many 
technological and managerial decisions during the design process in order to produce a best 
design. The ultimate goal of all such decisions is to minimize undesirable effects, while 
maximizing desirable effects, producing a “better, more efficient, less expensive solution that is 
in harmony with the laws of man and nature” (Ertas & Jones, 1993). 

Optimization typically occurs during the formulation of a design problem. According to Arora 
(1989), formulation of a problem requires approximately 50% of the total effort needed to solve 
it. Optimization techniques provide well-defined procedures to aid the designer in correctly 
formulating the problem. For example, Statnikov (1999) outlined three questions that designers 
should be able to answer when formulating a design problem: 

1. What to search for? (resulting in identifying the performance criteria.) 

2. Where to search? (resulting in defining all the constraints imposed on the design, 
which produces a set of feasible solutions.) 

3. How to search? (resulting in identifying the optimization technique that is most suited 
for the specific features of the problem being solved.) 

Formulating a design problem to achieve an optimal solution often involves transcribing a verbal 
description of the problem into a well-defined mathematical statement. This process enables the 
designer to search for the optimal design according to the identified performance criteria. 
Optimization methods frequently use mathematical concepts such as vector and matrix algebra, 
and calculus to analyze and optimize variables. As Arora (1989) pointed out, “the importance of 
proper formulation of a design optimization problem must be clearly understood because the 
optimum solution will only be as good as the formulation is” (p. 21). 

Purpose and Research Questions 

The purpose of the study was to assess the effectiveness of a unit of instruction in teaching core 
engineering concepts to secondary level technology education students. The following research 
questions guided the study. 

1. What type of engineering design activities and lessons will effectively deliver selected 
core engineering concepts to technology education students at the 10-12 grade level? 

2. Is there a relationship between performance in mathematics courses taken prior to 
participating in the unit of instruction and post-test instructional gain? 

3. Is there a relationship between performance in physical science courses taken prior to 
participating in the unit of instruction and post-test instructional gain? 



Methodology 
Research Design 

This study used a mixed methods quasi-experimental, pre-test post-test, no control group design, 
with the treatment as the independent variable and pretest-posttest as dependent variables. The 
participants in this study received a pre-test, treatment, and a post-test. Additionally, after the 
posttest, focus groups with randomly selected participants were conducted to provide qualitative 
data that allowed for a more in-depth analysis of the data. 

Treatment 

During the 2005-2006 school year, each technology education teacher that was involved in this 
study, including one mathematics and science teacher, completed 120 hours of professional 
development related to infusing engineering concepts into high school technology education. 
During this professional development, the eight technology education, one mathematics, and one 
science teacher helped to develop the 20-class session unit of instruction and the activities that 
supported infusing engineering concepts into the curriculum. These teachers were chosen 
because they were already participating in the NCETE professional development from which the 
study emerged. In addition, these teachers helped to solidify the treatment fidelity because they 
were key researchers in the development and delivery of the unit of instruction. The unit of 
instruction included four lessons, with specific content and activities. Other than the first lesson 
and activity (see below), teachers were permitted to teach the remaining lessons in any order. 

The unit of instruction was scheduled to be completed in 20 class sessions. Some of the teachers 
completed the unit prior to the 20 days, while other teachers went beyond the targeted number of 
days. Each lesson had a student version and teacher version. The teacher version of each lesson 
included supplemental materials including PowerPoint presentation materials specifically 
designed to address the key concepts being delivered in the unit. Grading rubrics were provided 
to the teachers and students for each lesson. Below is a description of each lesson and activity. 

The first lesson and activity (treatment) that the students (research participants) completed during 
this study to introduce COPA was called “Volume Barge.” In this lesson and activity, students 
were challenged to design and create a barge-type artifact made from one piece of 8.5” x 1 1” 
laminated card stock capable of holding the most weight before sinking; the barge had to be a 
rectangular shape. Students used volume calculations to optimize the best design based on the 
constraints. In a competitive style format, students graphed, using Microsoft Excel, the entire 
class performance to determine the winner. Using calculus based concepts, an optimum volume 
value was established, which set the standard for optimizing the design. This lesson and activity 
was deemed the favorite by most of the students from the eight schools, largely due to its 
competitive nature. 

A second lesson and activity the students completed was related to energy efficiency. During this 
lesson, students used mathematical formulas and existing data to determine R and U values for 
insulated wall cavities. Each group of students was provided with four completed wall sections, 
each having a 12” x 12” opening constructed with 2” x 4” and 2” x 6” framing materials. Three 
of the wall cavities were filled with different insulating materials, while the fourth section was 
left empty. The students were challenged to calculate the efficiency versus cost in a life-cycle 
approach, to determine the most optimal choice for insulation based on an average daily 
temperature and cost per thermal unit. Each completed wall section (four in all to create a 



square) was covered with a sheet of plywood, and a 100 Watt incandescent lamp was placed in 
the center. Using an infrared heat-sensing device, the students were able to determine insulation 
efficiency. From the experiences learned from this activity, the students had to use predictive 
analysis to optimize the life cycle costs of construction and building ownership over 5, 10, 15, 
and 25 year periods for a 2000 square foot structure. Students used Microsoft Excel to graph 
their results and present their results to the class. This lesson and activity was classified as the 
second favorite of most of the students from the eight schools, largely because it involved a 
hands-on experience. 

The third lesson and activity the students completed utilized a pre-fabricated golf ball launching 
device made from PVC and wood framing materials. Students used predictive analysis 
techniques to accurately launch a golf ball from a specific height and angle to a specific end 
distance. Students had to understand vectors, laws of motion, and energy to succeed. Students 
first predicted (non-analytic) how far the golf ball would travel and then used mathematical 
formulas to analyze how and where the ball would travel. In addition, students quickly learned 
that there exists an efficiency factor and that no machine or mechanism is 100% efficient. 
Students graphed their results and presented their findings to the class. This lesson and activity 
was rated as the third favorite by most of the students from the eight schools involved in this 
study, largely because the students could see the mathematical calculations in action in observing 
where the ball should and would land. 

The fourth lesson and activity the students completed dealt with identifying where and how 
mechanical energy is used and lost in their school. As an introductory activity to this lesson, 
students were provided with four different types of light (incandescent, fluorescent, halogen, and 
LED) to examine their efficiency. After classroom discussion regarding the cost, life-span, and 
energy used from these different sources of light, students became engaged in an activity where 
they had to locate four different sources of mechanical energy in their school to determine their 
efficiency. Students created a proposal that outlined where mechanical energy is being lost 
throughout their school and how they would use the wasted energy to complete productive work 
in other applications. Students used Newton scales, stop watches, and tape measures to determine 
mechanical energy (i.e., force, distance, and time). Each potential solution that students 
determined also included a wattage factor. During the presentation, students discussed their data 
collection methods, design solutions, the constraints associated with each design solution, and 
how each design would be optimized. Overall, students enjoyed this lesson and activity, but felt 
because it lacked a hands-on (building) approach, their engagement was not as high. 

Instrumentation 

The research team and a technology education teacher, who has a mechanical engineering 
degree, developed the test instrument used in this study. The development of the test instrument 
was guided by a review of the literature related to the engineering concepts: constraints, 
optimization, and predictive analysis. The thirty-item test instrument was developed to target the 
three concepts across three levels of Bloom’s taxonomy. For the purpose of this study. Bloom’s 
framework was reduced to three levels; comprehension, application, and analysis/synthesis. 
Comprehension included Bloom’s Knowledge and Comprehension categories, application 
included Bloom’s Application category, and analysis/synthesis combined Bloom’s Analysis, 
Synthesis, and Evaluation categories (Dalton & Smith, 1986). For example, questions written at 



the comprehension level used verbs offered by Dalton and Smith, including explain, predict, or 
discuss. 

Ten items were developed to target each of the three engineering concepts and were spread 
relatively evenly across the three levels of Bloom’s taxonomy. At the comprehension level there 
were four constraint, four optimization, and four predictive analysis questions. At the application 
level there were three constraint, three optimization, and three predictive analysis questions. At 
the analysis/synthesis level there were three constraint, three optimization, and three predictive 
analysis questions. 

The test was administered to the cohort of practicing and pre-service teachers during the summer 
professional development experience to estimate that the instrument was at the appropriate level 
of difficulty for secondary students and to identify any problematic questions. Their estimates 
were based on many years of experience of working with secondary level students. The research 
team and the technology education teacher then later refined potentially problematic questions. A 
further measure to ensure content validity was undertaken by sending the instrument to a panel 
of engineering and technology education professors and practitioners. These individuals 
completed a review of the instrument and a survey asking whether the questions measured an 
understanding of the three concepts at the different levels. Based on the feedback from the expert 
review panel, the test was further refined by the research team. This process resulted in an 
instrument containing thirty items that was believed to be at the appropriate difficulty level that 
measured an understanding of COPA at the three different levels. The reliability of the test 
instrument using Cronbach’s coefficient alpha was r = .782. 

Sampling Procedures 

A purposive sampling frame was utilized for this study, so the same teachers who developed the 
instructional materials were able to deliver the actual instruction in their classes to the study 
participants. Eight of the technology education teachers who participated in the NCETE 
professional development sessions at ISU recruited the students from their schools. Recruitment 
was conducted in nine intact technology education classes, since one teacher was able to recruit 
participants from two separate classes. Initially, 124 high school-level technology education 
students agreed to participate in the study. However, as a result of attrition, only 1 14 in = 1 14) 
students remained in the study at the time of the posttest. Within the final population there were 
102 male and 12 female students. 

In order to assess if there were significant differences between the subjects who remained in the 
study and those who did not, a one-way ANOVA at the a =.05 level was conducted using the pre- 
test scores as the dependent variable. The results suggested that there were not significant 
differences between the two groups in terms of test scores F(l,123) = .04, p > .05. 

Within twenty-four hours after completing the posttest, a focus group of systematic selection 
procedure was used to identify students from the eight high schools who participated in the 
study. Using the alphabetized course roster for each of the nine classes, every third student was 
selected to potentially participate in focus group. However, if that student did not wish to 
participate or had not submitted a consent form, that student was not selected for the sample. In a 
few cases, every third student resulted in a sample size of less than six, so two cycles of every 
third student, beginning with the last student selected, was conducted. For one school, the class 
size was small enough to conduct a focus group with all of the students. For two other schools, 



only a few students submitted consent forms or agreed to participate in the focus group, so all of 
those students participated in the focus group. A total of nine students from each school were 
selected to participate in the focus groups. The first six students selected were targeted as the 
primary participants, and three students were selected as alternates in case of absences or 
withdrawals. A total of eight schools and nine separate classes were involved with the focus 
group ( n = 54). Of the 54 students selected for the focus groups, 47 were males, 7 were females. 

Each focus group, lasting no longer than forty-five minutes, was guided by a script of fifteen 
questions that were divided into three categories: appeal questions, probing questions, and 
suggestions for improving the unit of instruction. Each focus group was conducted by two 
members of the research team. One of the researchers asked the questions and the other 
researcher acted as the scribe. In some cases, both the interviewer and scribe asked the students 
questions. Each focus group session was digitally recorded and saved as an MP3 file. Each of the 
researchers independently listened to each focus group session and developed synthesis 
paragraphs. 

Procedures 

This study was conducted in two phases. Phase One was the development of the 20-class session 
unit of instruction to be delivered to high school level technology education students, while 
Phase Two consisted of delivering and assessing the unit of instruction with the participants of 
this study. Phase One began during the Summer 2006 technology education session at ISU. 
Twelve high school level technology education teachers attended a five-day professional 
development session to develop the unit of instruction to integrate COPA in their technology 
education courses. Some of the teachers in this experience had previously attended ISU’s 
professional development sessions the previous year. Those sessions were also focused on 
integrating engineering concepts into high school level technology education curricula. 

During the first two days of the summer 2006 session, the teachers participated in presentations 
by technology education faculty members from ISU, an engineering professor from the 
University of Illinois at Urbana-Champaign (UIUC), NCETE doctoral fellows from UIUC; and 
science, mathematics, and technology education teachers from the public high school system. 
These presentations included a review of COPA, Wiggins and McTighe’s (2005)backward 
design process, and activities that emphasized the COPA concepts. 

During the third and fourth day of the workshop, the teachers working in groups of four 
developed the unit of instruction using the backward design process. As discussed in the 
instruction development section, each group of teachers developed artifacts for each of the 
different activities. Concurrently, the NCETE doctoral fellows, in conjunction with the 
technology education teacher from the high school system, began developing the test instrument 
to assess students understanding of COPA. 

On day five, the initial instrument was pilot tested with the high school teachers and graded. The 
results were shared with the teachers in order to obtain feedback regarding content and construct 
validity, and appropriate floor and ceiling height for high school level students. Additionally, the 
groups of teachers presented their activities to the session participants in order to obtain feedback 
that could be used to later enhance the activities. After the conclusion of the experience, the eight 



teachers who would actually be able to deliver the unit of instruction were asked to continue with 
refinement and final development of the activities. 


Between August and October, final lessons and activities related to Phase One were completed: 
(a) Institutional Review Board protocol approval was obtained from ISU and the UIUC, as well 
as from the individual high schools, (b) the test instrument was further refined after obtaining 
feedback from engineering, technology education professors, and practicing teachers, and (c) the 
eight teachers returned to ISU on October 5, 2006 for a one-day session to deliver the finalized 
units of instruction to the participating teachers involved in the study to receive formative 
feedback. Additionally during this time, technology education pre- service teachers enrolled at 
ISU began constructing the artifacts that were needed for the activities and pre-assembled these 
into kits that were sent to the participating high schools. The use of pre-assembled kits was 
deemed necessary in order to maintain treatment fidelity by making certain that all sites were 
using identical materials. In addition to uniformity provided by the pre-assembled kits, the 
process reduced the total time required to implement the study. This time element was an 
important factor, since teachers were injecting the research unit into their regular semester’s 
curriculum. 

Phase Two began during the first week of October 2006. Since the high school students were 
under 18, both parental consent and student assent were required. The technology education 
teachers distributed the appropriate forms to the participants; signatures were obtained and the 
forms were returned to their teachers who then forwarded the forms to the research team. The 
participants were then administered the pre-test that consisted of the 30 items related to 
measuring COPA at the three-levels of understanding. 

The delivery of the unit instruction by the teachers began between the second and third week of 
October 2006. Because the teachers were working within the limitations of their existing 
curriculum, it was not feasible for all sites to begin delivery of the unit of instruction on the same 
day. The unit of instruction was delivered over the span of 20 class sessions and immediately 
following the conclusion of delivering the 20 class session unit of instruction; the posttest was 
administered to the participants and returned to the research team. 

Findings and Discussion 

Consistent with the mixed model research design, data analysis consisted of quantitative and 
qualitative components. With the quantitative component, a series of dependent groups t-tests 
were conducted to explore differences between the pre-test and post-test. Student performance 
related to selected demographic variables was also examined. Qualitative data analysis consisted 
of examining the transcripts of post-instruction focus groups, which were conducted by the 
research team to explore students’ understanding of core concepts and the efficacy of the study’s 
unit of instruction. 

Quantitative Data Analysis 

A series of dependent groups t-tests were conducted to compare pre-test and post test scores. The 
initial analysis, which was conducted on the composite test scores, was followed by separate 
analyses of the instrument’s three dimensions (i.e., constraints, optimization, and predictive 



analysis). Student scores consisted of the number of items answered correctly of the instrument’s 
30 questions. A significant composite score gain of 3.22 was obtained between the pre-test and 
post-test (see Table 1). 

Table 1 

Dependent Groups t-testfor Composite Test Score 

Mean 

Pre-Test Score 14.74 

Post-Test Score 17.96 

77=114 

Total items on the test = 30 

While the gain scores were statistically significant, the overall percentage of items answered 
correctly was somewhat disappointing. Based on focus group discussion and interaction with 
participating teachers, low composite test scores were attributable to several factors. Among 
these factors were perceived test difficulty and voluntary participation in the study, where 
students were informed that test results would not be counted in their semester grades. In spite of 
relatively low test scores, focus group discussion, however, indicated that students clearly were 
able to identify the core engineering concepts selected for the study. In the aggregate, they also 
possessed a relatively sophisticated understanding of the interrelationship among the concepts. 
Focus group results suggest that the instruction may have been more effective than what the test 
scores indicate. 

In a focus group conducted with participating teachers, there was broad consensus that the test 
results were negatively impacted by the structure of the test. This included its high readability 
and conceptual levels as well as difficulties with knowledge transfer of examples used in the 
instrument and activities used during instruction. In spite of efforts made to validate the 
instrument during the planning stages of the study, it is clear that the study’s outcomes were 
influenced by these assessment issues. In spite of these issues, the teachers indicated that their 
students’ understanding of COPA concepts was clearly achieved as a result of the study. 

Separate dependent groups t-tests were also conducted for the instrument’s sub-scores (i.e., items 
assessing the three core engineering concepts). Mean score gains ranging from approximately 1 - 
1 Vi items were obtained with the highest gain score on the predictive analysis dimension. All 
three gain scores were statistically significant (see Table 2). 


Std. Dev. t Sig. 

4.872 8.604 .000 

4.984 



Table 2 

Dependent Groups t-testfor Core Concept Dimensions 



Mean 

Std. Dev. 

t 

Sig. 

Constraints (n=10) 

Pre-Test 

6.13 

2.106 

4.687 

.000 

Post-Test 

6.98 

2.018 



Optimization (n=10) 

Pre-Test Score 

4.22 

1.718 

5.513 

.000 

Post-Test Score 

5.12 

1.942 



Predictive Analysis (n=10) 

Pre-Test Score 

4.39 

2.151 

7.478 

.000 

Post-Test Score 

5.85 

2.019 




n = 114 


The instrument’s design also included the development of items at three levels of complexity 
along Bloom’s taxonomy of cognitive skills (comprehension, application, and 
analysis/synthesis). As could be anticipated, the highest net score gain occurred at the 
comprehension level (2.07) and the lowest score gain was obtained at the analysis/synthesis level 
(.53) (see Table 3). 

As noted above, focus group discussion indicates that the level of students’ understanding may 
exceed that suggested by the test score data. For example, a common observation of students was 
how the three engineering concept dimensions interact with one another in real world 
engineering design situations. Specifically, students commented on how optimized engineering 
designs routinely require tradeoffs among constraints (e.g., it is not possible to optimize all 
constraints; constraints tend to compete with one another). 


Table 3 

Dependent Groups t-testfor Conceptual Difficulty Levels 



Mean 

Std. Dev. 

t 

Sig. 

Comprehension (n=12) 

Pre-Test 

6.06 

2.557 

9.277 

.000 

Post-Test 

8.13 

2.533 



Application (n=9) 

Pre-Test Score 

4.62 

1.962 

3.613 

.000 

Post-Test Score 

5.25 

2.072 



Synthesis (n=9) 

Pre-Test Score 

4.05 

1.645 

2.893 

.005 

Post-Test Score 

4.58 

1.499 




n= 114 


The data were also analyzed to examine the possible effects of selected demographic factors on 
student learning. These factors included gender, ethnicity, and level of mathematics and science 



courses. Analysis of variance procedures, which were conducted on each of the variables, 
detected no statistically significant differences. 

Non-significant differences on the selected demographic variables are encouraging. Given the 
well-documented concern about female and minority involvement in scientific and engineering 
careers, this study’s results indicate that gender and ethnic differences may be minimized or 
reduced in controlled instructional situations. In other words, gender and ethnically-sensitive 
instructional design may facilitate learning of engineering concepts in ways that minimize 
demographic differences. The results of this study are encouraging, given the concern of many 
technology educators that the growing emphasis on engineering could reduce participation of a 
broad spectrum of students. 

Qualitative Data Analysis 

All of the focus group students commented on the amount of mathematics and science that was 
included within the unit of instruction and activities; predictive analysis throughout all of the 
activities was the least favorite aspect of the twenty-day unit of instruction. With the exception of 
one focus group, all of the students knew that constraints, optimization, and predictive analysis 
(COPA) were the key concepts being taught throughout the unit of instruction and provided 
examples of how and when they learned these concepts. As discussed in the treatment section, 
each lesson and activity targeted one or more of the key concepts. A majority of the students 
defined optimization as “the best solution to a problem, balancing trade-offs between competing 
factors.” Students defined predictive analysis as the “mathematical or scientific equations that 
are used before the artifact or problem is completed.” Students defined constraints as “the 
boundaries for what you can do and the parameters you have to stick to.” 

Of the three COPA concepts, predictive analysis was the most difficult to understand for the 
students and constraints was the easiest. A small minority of the students who completed the 
focus groups identified the COPA concepts as interconnected. Students used an analogy of the 
scientific method to COPA. Students were not familiar with optimization and predictive analysis 
before the treatment, but through their prior or existing technology education courses were 
fa mi liar with constraints. Students also commented that they take technology education courses 
because they are fun and activity-based, not mathematics or science-based. 

All of the focus group students rated the “Volume Barge” activity as their favorite. This activity 
focused on the concept of optimization. Furthermore, the activity challenged students to compete 
against their classmates for the best barge. A majority of the focus group students wanted more 
open design activities that were similar to the barge activity. 

Almost all of the focus group sessions revealed that students wanted a launching device, similar 
to a pneumatic powered device instead of the golf ball launching device that was used in the unit 
of instruction; students wanted a “boom” effect rather than the gravity fed device provided. 
Students seemed to like the wall insulation activity because it was more hands-on than the other 
activities within the unit. 

Overwhelming, the students in the focus groups commented on how mathematics and science 
concepts taught throughout the unit were better understood when they were connected to solving 



a problem or building an artifact. Students commented that they did not understand mathematics 
and science in their stand alone courses. They also commented on the positive nature of 
including most of the formulas they would need to solve for problems within student handouts or 
embedded within the activities. However, students commented that the theory of mathematics 
and science does not always translate into a properly working artifact. 

The last question to each focus group was whether or not this unit of instruction had influenced 
them to pursue post-secondary studies and a career in engineering or a related field. There was 
no indication that after completing the twenty-day unit of instruction that a positive or negative 
influence existed. 

Conclusions and Recommendations 

Based on the results of this study, some conclusions become apparent. It is clear that student 
learning was achieved as a result of students’ participation in the engineering design unit of 
instruction. While mean score gains from pre-test to post test were modest, they did indicate 
significant improvement in understanding of COPA concepts. Given the lack of significant 
gender, ethnic group, and mathematics/science background performance differences, the study 
also indicates that engineering concepts can be successfully delivered to a broad spectrum of 
students. These preliminary results are important since many technology educators are concerned 
that an engineering curricular focus could appeal to an academically capable subset of the 
technology education student body. 

Based on focus group discussions with students and teachers, some important factors emerged 
related to how engineering concepts were delivered in this study. These factors have important 
implications for future research, curriculum development, and professional development. One 
key factor has to do with an overt shift from procedural/activity -based curriculum and 
instruction, which as been typical for technology education, to an overt concept-based focus. The 
importance of this shift certainly extends beyond this research study or engineering curriculum. 

In this era of standards-based instruction, the technology education field must learn how to 
balance the historical appeal of engaging activities with curriculum development that is 
specifically designed to teach concepts (standards). While students generally indicated that they 
enjoyed the study’s activities, they also reported that they would have preferred to have actually 
constructed more of the devices used in the study (rather than having them pre-constructed in 
order to meet the time and treatment fidelity constraints of the study). 

Another significant challenge of research of this type has to do with the constraints involved with 
informed consent research. Focus group results indicate that student motivation to perform well 
in the study was eroded by their awareness that the test outcomes would not be included in their 
semester grades. 

Another important factor that emerged from the study had to do with the challenges associated 
with developing high quality, authentic assessments of COPA concepts. The outcomes of the 
study indicate that the test instrument was capable of detecting student learning at the various 
levels of conceptual difficulty. The psychometric properties were also sound, with acceptable 
levels of reliability and validity. Teacher involvement in the development and validation of the 



instrument used in this study was designed to ensure its appropriateness, including appropriate 
level of difficulty. However, focus group feedback indicated that students found the items to be 
demanding both in terms of reading level and conceptual load. While the multiple choice format 
provided objective data, future research should include more diverse and authentic formats. 

However, the findings of this study also indicated that there are specific areas of need in order to 
better develop these engineering concepts. For example, existing and pre- service technology 
education teachers need to be better equipped to develop and teach instruction focused on 
engineering design concepts. In particular, professional development focused on preparing 
technology education teachers to develop and teach instruction that is both concept-driven and 
activity-oriented is an area of need. Historically, technology education has focused primarily on 
procedural knowledge through hands-on activities that focus primarily on the artifact. In order to 
integrate engineering concepts within technology education, teachers need to develop 
pedagogical skills that include more focus on conceptual knowledge and the processes involved 
in engineering. 

Another area of need is the development of sound curriculum, activities, and assessments that 
target engineering design concepts. The instruction and activities developed for this study appear 
to have done an adequate job relaying the concepts to the students. However, with more 
refinement and focus these and similar activities could be used to teach engineering concepts 
even more effectively beyond the twenty-session research treatment. For example, separate units 
on constraints, optimization, and predictive analysis may help students better understand these 
engineering concepts. These activities need to maintain a hands-on component, which is an area 
of strength for technology education, because it appears to be a key to student motivation. In 
addition, authentic assessments need to be developed to assess student understanding. As 
revealed in this study, there are limitations to using tests to assess student learning of engineering 
concepts, especially at the analysis/synthesis level. Authentic assessments targeted at assessing 
student’s understanding need to be developed to gauge student learning. 
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